---
title: "Website Integration"
description: "Train your AI on any website content"
icon: "globe"
---

## Website Integration

Turn any website into a knowledge source for your AI:
- Crawl entire websites
- Extract structured content
- Regular content updates
- Custom crawling rules

### Setup Process

<Steps>
  <Step title="Add Website">
    Enter your website URL and crawling preferences
  </Step>
  <Step title="Configure Crawling">
    Set up crawling parameters:
    - URL patterns to include/exclude
    - Crawl frequency
    - Content selectors
  </Step>
  <Step title="Start Syncing">
    Begin the initial crawl and set up regular updates
  </Step>
</Steps>

### Features
- Smart content extraction
- Regular auto-updates
- Custom crawling rules
- SEO-friendly crawling 